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Abstract 

Teaching and assessing doctor-patient communication has become a priority in medical 
education. This pilot study evaluated resident physicians' perceptions of teaching and assessing 
doctor-patient communication skills related to pain management using a web-based format. 
Fifty-nine resident physicians completed four doctor-patient clinical encounters conducted with 
standardized patients (SPs) connected remotely via SKYPE. Quantitative and qualitative 
program evaluation data included residents' responses to Post-Exercise Surveys and their 
comments during focus group sessions. The investigation provided valuable information, 
especially regarding SP feedback received, the technology used and possible uses of this type of 
assessment in medical education. Future studies will focus on refinements in technology and 
alternative topics for use in remote clinical encounters. 


Introduction 

Historically, medical licensure and specialty board certifications have focused primarily on 
teaching and assessing medical knowledge, with lesser emphasis on other critical elements, 
such as doctor-patient communication and professionalism. Since 1999, more comprehensive 
medical competency frameworks have been integrated into the continuum of medical education 
by the ACGME and the AOA (Accreditation Council on Graduate Medical Education, 2007; 
American Osteopathic Association, 2004). For the 58,239 osteopathic physicians with active 
licenses in the United States (Young, Chaudhry, Rhyne and Dugan, 2011), such competencies 
include osteopathic principles and practice, patient care, medical knowledge, practice-based 
learning and improvement, interpersonal and communication skills, professionalism and 
systems-based practice. These competencies have been defined and formalized, emphasizing 
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that the competent physician should be educated and well-versed in a variety of domains, not 
just medical knowledge. The National Board of Osteopathic Medical Examiners (NBOME) has 
defined these competencies in their Fundamental Osteopathic Medical Competency Domains 
(FOMCD) document, delineating the specific elements for physician competence (National Board 
of Osteopathic Medical Examiners, 2011). 

This shift in focus also provides an opportunity to explore new teaching methods and 
assessment strategies. Numerous studies have cited the importance of teaching and assessing 
doctor-patient communication skills (Teutsch, 2003; Duffy, Gordon, Whelan, Cole-Kelly, & 
Frankel, 2004). Traditionally, training programs have relied on face-to-face interaction through 
the use of live patients, standardized patients (SPs) and Objective Structured Clinical 
Examinations (OSCEs) (Boulet, Smee, Dillon, & Gimpel, 2009). Currently, only 12.3% of 
residency training programs utilize standardized patient (SP) examinations, and only a small 
percentage utilize OSCEs (Holt, Miller & Nasca, 2010). These performance-based assessments 
provide opportunities to evaluate doctor-patient communication skills, but they are expensive 
and typically require learners to travel to the face-to-face encounters. 

To overcome these challenges some educators have explored web-based OSCEs or "Tele- 
OSCEs" as an adjunct for teaching doctor-patient communication. (Daetwyer, Cohen, Gracely, 
& Novak, 2010; Novack, Cohen, Peitzman, Beadenkopf, Gracely, & Morris, 2002). Such novel 
web-based training and assessments proved to provide meaningful communication skills 
instructions and assessments with greater flexibility for learners. 

Building on some of these findings, we decided to conduct a pilot study and built a similar 
program for residents to investigate the feasibility and user acceptance of this modality- 
formative encounters with remote standardized patients (RSPs) - using Skype as a cost 
effective video conferencing system. We chose pain management as the topic in focus 
because it is an issue of concern to all specialties and there are evidence-based strategies that 
combine communication skills and medical knowledge for addressing clinical problems. 
Furthermore, there is also an opportunity to address Osteopathic Manipulative Therapy-related 
issues. This article will summarize quantitative and qualitative feedback from the project 
participants, their views of the benefits and challenges of this educational method. 


Method 

For our pilot, we recruited osteopathic medical residents, a total of 59, randomly sampled, from 
a pool of 416 who responded to the Invitation to Participate letter that was emailed to all 
successful completers of COMLEX-USA Levels 2 CE and PE during the 2008 through 2011 
testing cycles (N= 18,520). Residents who completed the study were offered and received a 
$100 gift check and were entered into a raffle to win another $100 check. 

Nine RSPs were recruited, five from the East Coast and four from the West Coast. These RSPs 
were selected based on demographics, experience with OSCEs, comfort with technology, and 
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willingness to work from home. All received training in case portrayal, feedback delivery as well 
as the technologies to be used, from an experienced medical educator. 

Online sessions . Each resident participated in four 30-minute online sessions related to the 
assessment and counseling of pain. Each session consisted of the following components: 

• A 15-minute clinical encounter during which the resident physician's tasks were to obtain 
a patient history, address the patient's concerns regarding the reported pain and discuss 
treatment options; 

• A 5-minute "time-out" during which the resident completed a self-assessment survey 
while the RSP rated the resident's communication skills and recorded the content areas 
that were addressed; 

• A 10-minute immediate debriefing and feedback session guided by the RSP. 

The four clinical encounters included the following cases that were ambulatory in nature: chest 
pain (non-cardiac), headache following an automobile accident (referred pain), long-standing 
intermittent low back pain (opioid use/abuse) and rotator cuff shoulder pain (overuse injury). In 
advance of the remote encounter, the resident was provided with a Case Introduction that 
included the patient's chief complaint and the resident physician's task. The Case Introduction 
for the "chest pain" case is included in Appendix 1. 

Assessments. For each encounter, the following written assessments were completed right 
after each encounter: 

• Doctor-patient Communication Case-Specific Checklist consisting of 10 to 15 items, 
relating to the resident's medical history-taking during the encounter. It was completed 
by the RSP. A sample of the "chest pain" Case Checklist is included in Appendix 2. 

• Doctor-patient Global Communication Assessment, an NBOME-created Global 
Communication Assessment, reviewed and endorsed by NBOME's Clinical Skills Testing 
Advisory Committee. It is based on validated assessments such as Kalamazoo Essential 
Elements Communication Checklist (Adapted) (Joyce, Steenbergh, & Scher, 2010). It 
was completed by the RSP. The Global Communication Assessment is included in 
Appendix 3. 

• Doctor-Patient Communication Self-Assessment, using the same scale, rubric and 
instructions from the Global Communication Assessment described above. It was 
completed by the resident physician during the 5 minute "time-out" and is included in 
Appendix 4. Technology Report, a summary of the experiences with Skype and other 
technologies used during each doctor-patient encounter. This report, completed by the 
RSP, is included in Appendix 5. 

The completed assessments were emailed to the resident together with case-specific Teaching 
Points, within two days after the encounter to further enhance the learning experience. 

Following completion of all four encounters, participating residents were invited by email to 
complete an online Post-Exercise Survey designed to obtain opinions on training format, the 
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technologies used, the feedback provided and their overall experiences. The Survey consisted 
of 52 Likert-type items with four response options (Strongly Disagree, Disagree, Agree, Strongly 
Agree) arranged in 10 sections with open-ended comments permitted for each section, as well 
as an adjective checklist to capture global impressions. 

In addition, each participant was invited to join a one-hour focus group session that was 
conducted via conference call and was facilitated by an experienced medical educator. 

In summary, a variety of quantitative and qualitative measures were administered during the 
study. The Case Specific Checklist, the Global Communication Assessment and the Self- 
Assessment provided ratings of the residents' performance during the encounter, while the Post 
Exercise Survey and focus group responses provided information related to the residents' 
experience and acceptance of this formative assessment. As the stated purpose of this paper 
deals with medical residents' feedback on this web-based formative assessment, the results will 
concentrate on the analyses of the Post-Exercise Surveys and the focus groups. 

Results 

Sample. The participating sample consisted of 26 men and 33 women who represented 23 
different states, and who reported to have graduated from 22 different colleges of osteopathic 
medicine. They represented various medical specialties and training levels as shown in Table 1. 

Table 1. Demographics of Participating Sample of 59 Residents 


Gender 

56% female 


44% male 



Specialty 

27% Family Medicine 


20% Internal Medicine 


10% OB/GYN 


8% Emergency Medicine 


8% Pediatrics 


27% Other 



Training Level 

27% (1st year) 


41% (2 nd year) 


24% (3 rd year) 


8% (Other) 


Based on the AACOM Matriculate Report, the gender ratio as well as the race and ethnicity of 
the participant sample is fairly representative of the total 2009 - 2011 classes entering colleges 
of osteopathic medicine, with a slightly greater percentage of women participants. 
(http://www.aacom.org/data/applicantsmatriculants/Documents/2011Matriculantsummary.pdf). 
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Post-Exercise Survey. All 59 participating residents completed the Post-Exercise Survey. The 
residents' responses to the individual items, together with comments to each section are 
included in Appendix 6. A sample of the responses to the Survey's General Questions is 
included below in Table 2. 

Table 2. Selected Responses to General Questions from the Post-Exercise Survey 

from Participating Residents (n=59) 


General 

Strongly 

Disagree 

Disagree 

Agree 

Strong 

iy 

Agree 

N/A 

The web-based format was convenient for 
me. 

1 

(.017) 

1 

(.017) 

29 

(.491) 

28 

(.475) 

0 

This was a practical learning experience. 

1 

(.017) 

1 

(.017) 

29 

(.491) 

28 

(.475) 

0 

The web-based format was effective in 
teaching me communication skills. 

0 

6 

(.102) 

30 

(.508) 

23 

(.390) 

0 


Sample General Comments 

I find web based is much more difficult than in person. 

I had to learn to translate my skills into a manner that would accomplish the same thing 

over Skype. 

Talking to someone over Skype is nowhere near the same as in person. There are 
expressions and movements that are not conveyed over Skype. Also eliminating the physical 
exam prevents the doctor from asking questions during the exam. Every time there was a 

glitch with the audio it threw off my thought process. 

This was definitely more convenient than attending a scheduled lecture, however slightly 

less convenient than completing an online module. 

When the technology is not working perfectly, it also serves as a disruption and a 
distraction. 


As can be seen from Table 2, the overall response to the pilot study was very positive, with 
97% considering it to be a practical learning exercise. Ninety percent felt that that the web- 
based format was effective in teaching communication skills. 

Feedback provided to the residents by the RSP was an important component of the study. 
Based on the survey, virtually all participants (58 of 59) agreed or strongly agreed that the 
overall format for receiving verbal feedback from the RSP was valuable. One resident 
expressed the value of immediate feedback by stating "Immediate feedback was very effective. 
That's something that OSCEs at medical school did not offer." 

Many residents reported on having gained an improved ability to manage pain. Eighty-five 
percent of the participants agreed or strongly agreed that the encounters increased their ability 
not only to elicit a pain history from patients but also their comfort in doing so. Similarly, 86% 
felt that the exercises increased their ability to counsel patients about pain-related issues and 
treatment, while 90% reported an increased comfort in providing counseling. In addition, as 
osteopathic physicians, 73% agreed or strongly agreed that they felt confident in their ability to 
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describe Osteopathic Manipulative Treatment as a management option for reducing pain. 
Despite residents' support for the effectiveness of the exercises, quite a few of the comments 
expressed ambivalence toward the web format, stating that "it was difficult to be empathetic 
through a video conference" and "I felt like there was a disconnection." 

Table 3 illustrates a sample of residents' responses to the overall experience of the study. 

Table 3. Selected Responses to Overall Experience questions from the Post-Exercise 

Survey from Participating Residents (n=59) 


Overall Experience 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

N/A 

I prefer this web-based format to 

19 

28 

9 

3 

0 

traditional face-to-face clinical exercises. 

(.322) 

(.475) 

(.153) 

(.051) 


The amount of work required for this 
exercise was appropriate. 

0 

1 

(.017) 

46 

(.780) 

12 

(.203) 

0 

As a result of this exercise, I feel more 

3 

7 

37 

10 

2 

confident in my ability to communicate 
with my patients with regard to pain. 

(.051) 

(.119) 

(.627) 

(.169) 

(.034) 

I would recommend this communication 

2 

7 

34 

14 

2 

exercise to my colleagues. 

(.034) 

(.119) 

(.577) 

(.237) 

(.034) 

I would register for another web-based 

2 

3 

33 

19 

2 

exercise like this in the future. 

(.034) 

(.051) 

(.559) 

(.322) 

(.034) 

I was satisfied with the overall 

1 

2 

33 

22 

1 

experience (web-based exercise). 

(.017) 

(.034) 

(.559) 

(.373) 

(.017) 


While the large majority of participating residents, 80%, did not prefer this web-based format 
to traditional face-to-face clinical exercises, the same number reported that they felt more 
confident in their ability to communicate with patients regarding pain, providing support for the 
effectiveness of these exercises. Comments supported the preference for face-to-face 
encounters, stating "Regarding communication, the web based exercise was effective. In a true 
clinical scenario, it is still inferior to a face-to-face." In support for the web-based format, 88% 
stated that they would register for another web-based exercise like this in the future and the 
vast majority, 93%, stated that they were satisfied with the overall experience of this web- 
based exercise. 

Focus Group Discussions . Fifty-two of the residents participated in the 11 focus group sessions 
that were conducted within a 7 week time span. As indicated, the focus group sessions were 
facilitated by an experienced medical educator and followed a predefined framework of 
questions. Participants were informed that the discussion would be recorded for the purpose of 
transcription and further analysis. In conducting the analysis, each session transcript was 
analyzed regarding topics addressed and consistent themes were identified across the sessions. 

Across the 11 focus group discussions, several major themes emerged. The most common 
theme centered on the importance and value of the immediate verbal feedback offered by the 
RSPs which permitted interaction that written feedback does not allow. As stated by one 
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resident, "...once you get into residency, I don't know if we necessarily get feedback on our 
communication skills. It's much more about clinical decision making. So it was cool to get to 
remind yourself of how the patient is hearing you with your thought process and how you can 
improve." Many residents stated that communicating with a patient via Skype was more difficult 
initially, but as the encounters progressed, they became more comfortable and felt more 
effective as they were better able to use the feedback they had been given in previous 
encounters. One resident stated, "Probably the best thing I took away from this experience, 
the verbal feedback from the patients about my interviewing style." 

A second important theme related to the residents' emphasis of the importance of hands-on 
physical exams for history taking and diagnosis, particularly for osteopathic physicians. As one 
resident stated, "...as an osteopath, one of our principles is to put your hand on the patient." 
As stated by another, "If we ever practice medicine like this, really it would be sad. There are 

so many things you pick up on during physical exams from patients I just think this would be 

a bad way to practice medicine but as far as training people on how to take histories, it's a 
good way on how to be more compassionate." 

Another theme voiced by participants was interest in telemedicine and wanting to learn more 
about its use in medicine's future. One resident stated that these encounters "...got me a little 
more interested in telemedicine and realizing that it's not just a phone call. ...on the other side, 
there's a patient and they're having problems and it's not just like a computer you're talking to." 
Residents proposed other topics that would be appropriate to a web-based format including 
breaking bad news to a patient, counseling topics such as smoking cessation and psychiatric 
topics such as depression. 

For most residents, it was a valuable experience: "I really didn't think I was going to gain as 
much as I've gained. I thought it was just a matter of participating in a research program. I've 
learned so much, I've learned what my deficiencies are; it contributed a lot to my education." 

To summarize, the results of the Post-Encounter Survey and the focus group discussions are 
very supportive of each other. Both emphasized the value and appreciation of immediate 
verbal feedback, the value of this online format as a learning experience but not as a 
replacement for an in-person clinical exam. 

Discussion 

As a pilot, this study presented valuable information regarding the possible use of such web- 
based learning in medical education. This study was innovative in many ways. It investigated 
the novel use of technology, Skype, in a web-based educational module that taught doctor- 
patient communication skills using formative assessments. It developed the concept and role of 
the RSP and included the presentation of not just written feedback, but immediate verbal 
feedback as well, something that is often neglected in the medical education process. 
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However, the study was conducted primarily as a pilot, with a novel approach and an eye 
toward next steps. While there were many aspects of this study that worked very well, the 
success of some elements could be improved. 

In addition, while the results of the Post-Exercise Survey suggested that most RSPs and 
residents considered Skype easy to use, there were nevertheless technology challenges that 
occurred for various reasons. Some of the most common problems included dropped calls, poor 
video quality due to high Skype volume and poor audio quality resulting in an annoying "echo," 
which was distracting. Some residents reported difficulties with visualizing subtle expressions. 
Future studies may consider alternatives to Skype. Since participation in the study was 
voluntary there is a possible selection bias toward residents more open to novel technologies. 
Furthermore, the sample size was small and limited to osteopathic residents; a larger and more 
diverse sample could provide more generalizability. 

Nevertheless, despite the challenges and limitations, this pilot provided support for continuing 
research in the use of web-based OSCEs as a formative assessment for doctor-patient 
communication, offering an educational format that is practice relevant. Future research will 
focus on considering other topics, particularly counseling-related, for the teaching and formative 
assessment of doctor-patient communication skills. A more in-depth analysis of the content of 
the focus group discussion is planned, together with the inclusion of additional raters to confirm 
inter-rater consistency. Additional exploratory studies are planned with practicing physicians to 
address continuous medical certification requirements. 


References 

Accreditation Council on Graduate Medical Education. (2007). Common program requirements: 
general competencies. Retrieved from 

http://www.acgme.org/Outcome/comp/GeneralCompetenciesStandards21307.pdf . (Archived on 
8 June 2011 at http://www.webdtation.org/5zIFGOLtn) 

American Association of Colleges of Osteopathic Medicine. (2012). AACOMAS Matriculant 
Profile. 2011 Entering Class. Retrieved from 

(http://www.aacom.org/data/applicantsmatriculants/Documents/2011Matriculantsummary.pdf). 

American Osteopathic Association. (2004). Core Competency Compliance Program (CCCP) Part 
III. Retrieved from http://www.do-online.org/pdf/acc_cccppart3.pdf (Archived on 16 May 
2012 at http://www.webcitation.org/67i2YR4Sg) 

Boulet, J.R., Smee, S.M., Dillon, G.F., & Gimpel J.R. (2009). The use of standardized patient 
assessments for certification and licensure decisions. Simulation in Healthcare, 4, 35- 42. doi: 
10.1097/SIH.0b013e318182fcbc. 


9 


Daetwyer, C.F., Cohen, D.G., Gracely, D., & Novak, D.N. (2010). eLearning to enhance 
physician patient communication: a pilot test of "doc.com" and "WebEncounter" in teaching bad 
news delivery. Medical Teacher, 32, e374-e383. doi: 10.3109/-142159x.2010.495759. 

Duffy, F.D., Gordon, G.H., Whelan, G., Cole-Kelly, K., & Frankel, R. (2004). Assessing 
competence in communication and interpersonal skills: the Kalamazoo II report. Academic 
Medicine, 79(6), 495-507. 

Holt, K.D., Miller, R.S., & Nasca, T.J. (2010) Residency programs evaluations of the 
competencies: data provided to the ACGME about types of assessments use by programs. 
Journal of Graduate Medical Education, 2(4):649-55. doi: 10.4300/JME-02-04030. 

Joyce, B.L., Steenbergh, T., Scher, E. (2010). Use of the Kalamazoo essential elements 
communication checklist (adapted) in an institutional interpersonal and communication skills 
curriculum. Journal of Graduate Medical Education, 2(2): 165-9. 

National Board of Osteopathic Medical Examiners. (June 2011). Fundamental Osteopathic 
Medical Competency Domains Document. Retrieved from 

http://www.nbome.org/docs/NBOME%20Fundamental%200steopathic%20Medical%20Compet 
encies.pdf (Archived on 16 May 2012 at http://www.webcitation.org/67i2FI7pN) 

Novack, D.FI., Cohen, D., Peitzman, S.J., Beadenkopf, S., Gracely, E., & Morris, J. (2002). A pilot 
test of WebOSCE: a system for assessing trainees' clinical skills via teleconference. Medical 
Teacher, 24(5), 483-7. doi: 10.1080.0142159021000012504. 

Teutsch, C. (2003). Patient-doctor communication skills. Medical Clinics of North America, 87, 
115-1145. 

Young, A., Chaudhry, H., Rhyne, J., fk fk Dugan, M. (2011). A census of actively licensed 
physicians in the United States. Journal of Medical Regulation, 96(4), 10-20. 



10 


Appendix 1 Case Introduction for the Chest Pain Case 

CASE 101 P: Case Introduction 


Patient Name 

Linda Smith 

Age 

48 yo 

Setting of Visit 

Primary Care Office 

Background 

This is this first time you are seeing the patient. She has no 
insurance and is “self-pay.” Given the patient is new to your 
practice, past medical history is unknown to you. 

Chief Complaint 

“1 have a pain in my chest and I’ve got 2 kids to take care of.” 

Vital Signs 

Resp Rate 

18 

Heart Rate 

104 

Blood Press 1 36/82 


Physician Task 

1. Obtain a pain history 

2. Address the patient’s concerns and 
pain’s impact on daily living 

3. Discuss further evaluation and/or 
treatment options 


*Note: You are not to conduct a physical exam. Therefore, please refrain from asking the patient to 
disrobe, perform range of motion testing, or conduct other physical examination maneuvers. 
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Appendix 2 Chest Pain Checklist 


Case 101P Chest Pain 


Physician: 

RSP: 

Date: 


Time: 

No Yes Comments 

Checklist Items 


1 . Location (e.g. “Where is the pain located?”) 






2. Palliation (e.g. “What do you do to make the pain 
better?”) 






3. PrOVOCatiOn (e.g. “What makes the pain worse?”) 






4. Quality (e.g. “What is the pain like?” “Is it dull or sharp?”) 






5. Radiation (e.g. “Does the pain radiate anywhere?”) 






6. SGVGrity (e.g. “On a scale of 1-10, how would you rate 
the pain?”) 






7. Associated Symptoms (e. g . “Any other 

symptoms with the pain?” “Any dizziness, nausea?”) 






8. TiminQ (e.g. “How long has the pain been present?” “Is it 
constant?”) 

9. PMH (e.g. “Do you have any medical problems?”) 

io. Impact on daily living (e. g . “what are you 

worried about?”) 

ii. Diagnosis/prognosis (e.g. The pain is most 

likely musculoskeletal, not heart related”) 















12. Alternative treatments (e.g. “Treatments may 

include OMT, physical therapy, rest”) 
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Appendix 3 Doctor-patient Global Communication Assessment 


Global Communication Assessment (Continuous Professional Development)* 

PHYSICIAN'S NAME: RATER'S NAME: CASE NUMBER: 

1. BUILDS THE 
RELATIONSHIP 

□ 1 HEEDS MUCH IMPROVEMENT ADEQUATELY □ 4 DONE EXCELLENTLY 

• Doesn't express care or concern 

(verbally &/or non-verbally) • Uses words and non-verbal cues that 

• Does not adjust tone or pace to demonstrate care & concern 

patient's needs • Adjusts tone and pace to patient's needs 

• Interrupts or redirects frequently • Listens, encourages patient's 

• Shows no interest in psychosocial participation, and redirects appropriately 

issues, only focuses on biomedical • Shows interest in psychosocial and 

issues biomedical issues 

• Fails to share talk time; verbally • Shares talk time; no verbal dominance 

dominates 


2. OPENS THE 
DISCUSSION 

□ 1 NEEDS MUCH IMPROVEMENT 

□ 2 NEEDS SOME 
IMPROVEMENT 

□ 3 DONE 
ADEQUATELY 

□ 4 DONE EXCELLENTLY 

(BEGINNING OF 

• Interrupts patient's opening 



• Allows patient to complete opening 

INTERVIEW) 

statements 



statement without interruption 


• Does not elicit full set of concerns 



• Asks “anything else?” to elicit full set of 


• Does not summarize goals for the 



concerns 


visit 



• Summarizes goals for visit 


3. GATHERS 
INFORMATION 


□ 1 NEEDS MUCH IMPROVEMENT 

□ 2 NEEDS SOME 
IMPROVEMENT 

□ 3 DONE 
ADEQUATELY 


□ 4 DONE EXCELLENTLY 


• 

Does not balance open-ended and 



• 

Balances open-ended and close-ended 



close-ended questions 




questions 


• 

Ignores patient's expression or clues 



• 

Responds to verbal & nonverbal clues 



about concerns 



• 

Refocuses the interview as important 


• 

Prematurely narrows the focus of 




biomedical or psychosocial information 



the interview 




arise 


• 

Flow is illogical and disorganized 



• 

Flow is logical and organized 


• 

Doesn't indicate transitions 



• 

Indicates transitions 


• 

Doesn't summarize 



• 

Summarizes and gives patient opportunity 
to correct or add information 


4. 

UNDERSTAND 

□ 1 NEEDS MUCH IMPROVEMENT 

□ 2 NEEDS SOME 
IMPROVEMENT 

□ 3 DONE 
ADEQUATELY 

□ 4 DONE EXCELLENTLY 

THE PATIENT’S 

• Doesn't elicit patient's beliefs, 



• Elicits patient's beliefs, concerns & 

PERSPECTIVE 

concerns & expectations about 



expectations about illness & treatment 


illness & treatment 



• Acknowledges and validates patient's 


• Doesn't acknowledge and validate 



perspective 


patient's perspective 



• Asks about patient's understanding of 


• Doesn't ask about patient's 



illness or situation 


understanding of illness or situation 





5. SHARES 
INFORMATION 

i—i * Mccnc mi iru imddih/cmcmt □ 2 NEEDS SOME □ 3 DONE 

□ 1 NEEDS MUCH IMPROVEMENT IMPROVEMENT ADEQUATELY 

□ 4 DONE EXCELLENTLY 


• Uses medical jargon & gives overly 

• Explains using words patient can 


technical information 

understand 


• Doesn't explain symptoms, 

• Gives clear explanation of symptoms, 


diagnostic tests, diagnosis and/or 

diagnostic tests, diagnosis and/or 


treatment options 

treatment options 


• Doesn't verify patient's 

• Verifies patient's understanding of the 


understanding of information 

information standing of illness or situation 
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6. REACHES 
AGREEMENT 

□ 1 NEEDS MUCH IMPROVEMENT 

□ 2 NEEDS SOME 
IMPROVEMENT 

□ 3 DONE 
ADEQUATELY 

□ 4 DONE EXCELLENTLY 


• Makes decisions for patient without 
discussion of patient's role 

• Does not discuss pros/cons of 
options & uncertainties 

• Does not verify patient's 
understanding of options 

• Does not explore patient's 
preferences 

• Does not propose mutually 
agreeable decision 



• Discusses patient's role in decision making 

• Discusses pros/cons of options & uncertainties 
associated with decision 

• Verifies patient's understanding of options 

• Explores patient's preferences 

• Proposes mutually agreeable decision 


7. PROVIDES 
CLOSURE 

□ 1 NEEDS MUCH IMPROVEMENT 

□ 2 NEEDS SOME 
IMPROVEMENT 

□ 3 DONE 
ADEQUATELY 

□ 4 DONE EXCELLENTLY 


Doesn't ask about remaining 
questions 

Doesn't summarize next steps or 
follow-up 



• Asks if patient has any remaining questions 

• Summarizes next steps and follow-up 


8. OVERALL 

PATIENT 

SATISFACTION 

□ 1 NOT SATISFIED 

□ 2 SOMEWHAT 
SATISFIED 

(would prefer not to 
return to this physician or 
recommend him/her to 
family/friends) 

□ 3 MOSTLY 
SATISFIED 

(would be willing to 

□ 4 VERY SATISFIED 


(would not return to this physician or 
recommend him/her to family/friends) 

return to this physician 
and may recommend 
him/her to 
family/friends) 

( would definitely return to this physician and strongly 
recommend him/her to family/friends) 


IRREGULARITIES 
AND RATING 
DIFFICULTIES 


GENERAL 
COMMENTS AND 
REFLECTIONS 



‘Informed by Essential Communication Elements, Kalamazoo Consensus Statement. Duffy FD, Gordon GH, Whelan G, Cole- 
Kelly K, Frankel R. Assessing competence in communication and interpersonal skills: the Kalamazoo II report. Academic 
Medicine. 2004;79(6):495-507. 
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Appendix 4 -Doctor-Patient Communication Self-Assessment 



YOUR NAME: 
DATE: 


Physician Self-Assessment: Global Communication Assessment (Continuous Professional Development)* 

CASE NUMBER: 


1. BUILDS THE 
RELATIONSHIP 

□ 1 NEEDS MUCH IMPROVEMENT 

□ 2 NEEDS SOME 
IMPROVEMENT 

□ 3 DONE ADEQUATELY 

□ 4 DONE EXCELLENTLY 


• Doesn't express care or concern 
(verbally &/or non-verbally) 

• Does not adjust tone or pace to 
patient's needs 

• Interrupts or redirects frequently 

• Shows no interest in psychosocial 
issues, only focuses on biomedical 
issues 

• Fails to share talk time; verbally 
dominates 


• 

• 

• 

• 

• 

Uses words and non-verbal cues that 
demonstrate care & concern 
Adjusts tone and pace to patient's 
needs 

Listens, encourages patient's 
participation, and redirects 
appropriately 

Shows interest in psychosocial and 
biomedical issues 

Shares talk time; no verbal dominance 


2. OPENS THE 
DISCUSSION 

□ 1 NEEDS MUCH IMPROVEMENT 

□ 2 NEEDS SOME 
IMPROVEMENT 

□ 3 DONE ADEQUATELY 

□ 4 DONE EXCELLENTLY 

(BEGINNING OF 
INTERVIEW) 

Interrupts patient's opening 
statements 

• Does not elicit full set of concerns 
Does not summarize goals for the 
visit 


• 

• 

• 

Allows patient to complete opening 
statement without interruption 
Asks “anything else?” to elicit full set 
of concerns 

Summarizes goals for visit 


3. GATHERS 
INFORMATION 

□ 1 NEEDS MUCH IMPROVEMENT 

□ 2 NEEDS SOME 
IMPROVEMENT 

□ 3 DONE ADEQUATELY 

□ 4 DONE EXCELLENTLY 


• Does not balance open-ended and 
close-ended questions 

• Ignores patient's expression or 
clues about concerns 

• Prematurely narrows the focus of 
the interview 

• Flow is illogical and disorganized 

• Doesn't indicate transitions 

• Doesn't summarize 


• 

• 

• 

• 

• 

• 

Balances open-ended and close- 
ended questions 

Responds to verbal & nonverbal clues 
Refocuses the interview as important 
biomedical or psychosocial 
information arise 
Flow is logical and organized 
Indicates transitions 
Summarizes and gives patient 
opportunity to correct or add 
information 


□ 1 NEEDS MUCH IMPROVEMENT 

□ 2 NEEDS SOME 
IMPROVEMENT 

□ 3 DONE ADEQUATELY □ 4 DONE EXCELLENTLY 

• Doesn't elicit patient's beliefs, 


• Elicits patient's beliefs, concerns & 

concerns & expectations about 


expectations about illness & 

illness & treatment 


treatment 

• Doesn't acknowledge and validate 


• Acknowledges and validates patient's 

patient's perspective 


perspective 

• Doesn't ask about patient's 


• Asks about patient's understanding 

understanding of illness or situation 


of illness or situation 


4. UNDERSTAND 
THE PATIENT’S 
PERSPECTIVE 


15 


5. SHARES 
INFORMATION 

□ 1 NEEDS MUCH 
IMPROVEMENT 

□ 2 NEEDS SOME 
IMPROVEMENT 

□ 3 DONE ADEQUATELY □ 4 DONE EXCELLENTLY 


• Uses medical jargon & gives 


• Explains using words patient can 


overly technical information 


understand 


• Doesn't explain symptoms, 


• Gives clear explanation of symptoms, 


diagnostic tests, diagnosis 


diagnostic tests, diagnosis and/or 


and/or treatment options 


treatment options 


• Doesn't verify patient's 


• Verifies patient's understanding of 


understanding of information 


the information 


6. REACHES 
AGREEMENT 

□ 1 NEEDS MUCH 
IMPROVEMENT 

□ 2 NEEDS SOME 
IMPROVEMENT 

□ 3 DONE ADEQUATELY □ 4 DONE EXCELLENTLY 


• Makes decisions for patient 


• Discusses patient's role in decision 


without discussion of patient's 


making 


role 


• Discusses pros/cons of options & 


• Does not discuss pros/cons of 


uncertainties associated with 


options & uncertainties 


decision 


• Does not verify patient's 


• Verifies patient's understanding of 


understanding of options 


options 


• Does not explore patient's 


• Explores patient's preferences 


preferences 


• Proposes mutually agreeable 


• Does not propose mutually 


decision 


agreeable decision 




7. PROVIDES 
CLOSURE 

□ 1 NEEDS MUCH 
IMPROVEMENT 

□ 2 NEEDS SOME 
IMPROVEMENT 

□ 3 DONE ADEQUATELY 

□ 4 DONE EXCELLENTLY 


• Doesn't ask about remaining 
questions 

Doesn't summarize next steps 
or follow-up 



• Asks if patient has any remaining 
questions 

• Summarizes next steps and follow-up 


8. OVERALL 

PATIENT 

SATISFACTION 

□ 1 NOT SATISFIED 

(The patient would not return to me or 
recommend me to family/friends) 

□ 2 SOMEWHAT 
SATISFIED 

(The patient would most 
likely not return to me or 
recommend me to 
family/friends) 

□ 3 MOSTLY SATISFIED 

(The patient would be willing to 
return to me and may recommend 
me to family/friends) 

□ 4 VERY SATISFIED 

(The patient would definitely return to me and 
strongly recommend me to family/friends) 


GENERAL 

COMMENTS 


REFLECTIONS 



‘Informed from Essential Communication Elements, Kalamazoo Consensus Statement. Duffy FD, Gordon GH, Whelan G, Cole- 
Kelly K, Frankel R. Assessing competence in communication and interpersonal skills: the Kalamazoo II report. Academic 
Medicine. 2004;79(6):495-507. 
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Appendix 5 Technology Report 

Encounter Technology Report 

The information below is requested as a record of any technology difficulties or other 
irregularities you encountered during your session. 

RSP Name: Case #: 


Resident Name: Date/Time: 


Did record the encounter? 

□ yes 

□ no 

Did record the feedback session? 

□ yes 

□ no 

Did you upload the recordings? 

□ yes 

□ no 


Did the encounter any technical difficulties during the encounter? □ yes 

□ no 

Comments / Irregularities: 


Did you have any difficulty completing the Checklist and Global Assessment forms? 

□ yes □ no 

Comments/ Irregularities: 


Did you encounter any difficulties with the feedback session? □ yes □ no 

Comments / Irregularities: 


Thank you again for participating in this innovative research project! If you have questions or need further support, please let us 
know. 
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Appendix 6 Responses to physician post-exercise survey (n=59) 


General 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

N/A 

The web-based format was convenient for me. 

1 

1 

29 

28 

0 

This was a practical learning experience. 

1 

1 

29 

28 

0 

The web-based format was effective in teaching me 
communication skills. 

0 

6 

30 

23 

0 

Technology 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

N/A 

I am familiar with Skype (e.g., I have used Skype 
before this exercise). 

5 

8 

22 

24 

0 

I am familiar with other video conferencing 
programs (similar to Skype). 

6 

19 

18 

14 

2 

1 found Skype easy to use. 

0 

1 

25 

33 

0 

1 found the Technology Instructions For Physicians 
document useful for completing the exercise. 

0 

1 

28 

23 

7 

1 found the “Technology Check’’ (call in hours) with 
the NBOME helpful. 

0 

1 

10 

12 

36 

1 found the NBOME staff helpful when addressing 
technology-related issues 

0 

2 

9 

11 

37 

I did not experience any technical difficulties during 
the encounters. 

11 

23 

15 

8 

2 

Scheduling 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

N/A 

Scheduling encounters was easy. 

0 

2 

25 

32 

0 

Email was an effective communication tool for 
scheduling the encounter. 

0 

1 

20 

38 

0 

Staff members responsible for scheduling were 
professional. 

0 

0 

12 

47 

0 

Remote Standardized Patient 
Experience 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

N/A 

I was able to communicate easily with the Remote 
Standardized Patients (RSPs). 

0 

4 

39 

16 

0 

I was able to get the information I needed from the 
Remote Standardized Patients (RSPs). 

0 

4 

35 

20 

0 

Exercises increased my awareness of addressing 
patients’ concerns related to pain. 

1 

10 

29 

19 

0 

Exercises increased my ability to elicit a 
comprehensive pain history. 

1 

8 

35 

15 

0 

Exercises increased my comfort in eliciting a 
comprehensive pain history from patients. 

1 

7 

36 

14 

1 

Exercises increased my ability to counsel patients 
about pain-related issues and treatment. 

1 

6 

40 

11 

1 

Exercises increased my comfort in counseling 
patients about pain-related issues and treatment. 

1 

5 

42 

11 

0 




18 


I felt confident in my ability to describe Osteopathic 
Manipulative Treatment (OMT) as a management 
option for reducing pain. 

0 

12 

28 

15 

4 

Remote Standardized Patients (RSPs) were 
professional. 

0 

0 

22 

37 

0 

Believability of Cases 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

N/A 

The portrayal of the Chest Pain Case was believable. 

0 

1 

34 

24 

0 

The portrayal of the Headache Case was believable. 

0 

1 

32 

26 

0 

The portrayal of the Low Back Pain Case was 
believable. 

0 

0 

29 

30 

0 

The portrayal of the Shoulder Pain Case was 
believable. 

1 

5 

31 

22 

0 

Verbal Feedback 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

N/A 

The overall format for receiving verbal feedback 
(from the Remote Standardized Patient) was 
valuable. 

0 

1 

32 

26 

0 

The verbal feedback I received from the Chest Pain 
Case was valuable. 

1 

1 

30 

27 

0 

The verbal feedback I received from the Headache 
Case was valuable. 

0 

4 

33 

22 

0 

The verbal feedback I received from the Low Back 
Pain Case was valuable. 

0 

2 

31 

26 

0 

The verbal feedback I received from the Shoulder 
Pain Case was valuable. 

1 

6 

28 

24 

0 

Written Assessment (Global Assessment) 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

N/A 

The process of completing the self-assessment 
forms was valuable. 

1 

11 

39 

8 

0 

Information provided in the global communication 
assessment (completed by the RSP) was valuable. 

0 

1 

37 

21 

0 

Information provided in the global communication 
assessment (completed by the RSP) was an accurate 
assessment of my performance during the exercise. 

0 

3 

42 

14 

0 

Written Assessment (Checklist) 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

N/A 

Information provided in the Chest Pain Checklist 
was valuable. 

0 

3 

40 

16 

0 

Information provided in the Chest Pain Checklist 
was an accurate assessment of my performance 
during the encounter. 

0 

3 

41 

15 

0 

Information provided in the Headache Checklist 
was valuable. 

0 

3 

42 

14 

0 

Information provided in the Headache Checklist 
was an accurate assessment of my performance 
during the encounter. 

0 

5 

42 

12 

0 

Information provided in the Low Back Pain 
Checklist was valuable. 

0 

2 

42 

15 

0 
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Information provided in the Low Back Pain 
Checklist was an accurate assessment of my 
performance during the encounter. 

0 

4 

41 

14 

0 

Information provided in the Shoulder Pain Checklist 
was valuable. 

0 

2 

41 

16 

0 

Information provided in the Shoulder Pain Checklist 
was an accurate assessment of my performance 
during the encounter. 

1 

3 

42 

13 

0 

Teaching Points 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

N/A 

The Chest Pain Teaching Points were valuable. 

0 

5 

35 

19 

0 

The Headache Teaching Points were valuable. 

0 

4 

36 

19 

0 

The Low Back Pain Teaching Points were valuable. 

0 

5 

35 

19 

0 

The Shoulder Pain Teaching Points were valuable. 

0 

7 

33 

19 

0 

Overall Experience 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

N/A 

I prefer this web-based format to traditional face-to- 
face clinical exercises. 

19 

28 

9 

3 

0 

The amount of work required for this exercise was 
appropriate. 

0 

1 

46 

12 

0 

As a result of this exercise, I feel more confident in 
my ability to communicate with my patients with 
regard to pain. 

3 

7 

37 

10 

2 

I would recommend this communication exercise to 
my colleagues. 

2 

7 

34 

14 

2 

I would register for another web-based exercise like 
this in the future. 

2 

3 

33 

19 

2 

I was satisfied with the overall experience (web- 
based exercise). 

1 

2 

33 

22 

1 




